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INTRODUCTION. 


"Bayesian"  reliability  tests  have  been  a  much  discussed  but  seldom  ap¬ 
plied  engineering  technique.  Part  of  the  problem  is  a  lack  of  understanding  of 
the  procedure,  compounded  by  the  promulgation  of  controversial  approaches. 
However,  even  an  approach  well  understood  and  accepted  by  the  user  may  fall  by 
the  wayside  because  of  mathematical  complexity.  It  is  the  purpose  of  this 
report  to  provide  an  understandable,  easy  to  use,  and  believable  approach  to 
"Bayesian"  reliability  testing  in  order  to  encourage  its  practical  applica¬ 
tion. 


"Bayesian"  methods  for  reliability  demonstration  are  procedures  for  in¬ 
corporating  existing  data  into  a  reliability  demonstration  plan.  The  term 
stems  from  Bayes'  theorem  of  conditional  probability  and,  strictly  speaking, 
not  all  reliability  test  plans  using  prior  data  are  correctly  called  Bayesian 
Reliability  Tests.  Hence,  the  quotation  marks  in  the  title  of  the  report.  For 
editorial  simplicity,  we  will  discontinue  the  use  of  the  quotation  marks  and 
ask  the  reader  to  accept  the  term  as  referring  to  all  test  plans  based  on  the 
use  of  prior  data. 

The  interest  in  Bayesian  test  plans  arises  from  its  potential  to  permit 
shorter  reliability  tests  at  no  loss  of  confidence.  However,  this  advantage 
holds  only  If  the  prior  is  favorable,  i.e.  existing  data  on  the  type  of 
equipment  to  be  tested  indicates  a  mean  time  between  failure  (MTBF)  higher  than 
required.  It  is  quite  possible  for  a  Bayesian  approach  to  result  in  a  longer 
test  than  classical  test  methods  if  the  prior  is  unfavorable.  This  point  must 
be  considered  if  the  intent  is  strictly  to  reduce  test  time. 

A  less  obvious  advantage  to  Bayesian  tests  is  the  enrichment  of  the 
choice  of  statistical  risks.  In  classical  test  plans,  the  statistical  risks 
are  the  consumer's  risk  (the  probability  of  accepting  an  equipment  If  its 
actual  MTBF  equals  a  specific  value  defined  as  undesired)  and  the  producer's 
risk  (the  probability  of  rejecting  an  equipment  if  the  actual  MTBF  equals  a 
specific  value  defined  as  desired).  Using  Bayesian  test  plans,  several  other 
definitions  of  risk  are  possible,  resulting  in  a  selection  of  those  risks  most 
meaningful  to  the  circumstances. 


In  the  following  sections  we  will  compare  the  Bayesian  approach  to  class¬ 
ical  reliability  demonstration,  examine  some  of  the  Bayesian  tools  available, 
and  develop  a  suggested  approach. 

2.  BAYESIAN  VS.  CLASSICAL  PLANS 

Figure  1  shows  the  flow  for  developing  a  classical  test  plan.  The 
responsible  engineer  starts  by  determining  what  values  of  producers  and  con¬ 
sumers  risks  are  acceptable  to  him.  He  has  no  choice  of  the  type  of  risks 
Involved,  only  of  their  values.  This  of  course  implies  defined  values  for  a 
desired  MTBF  (the  upper  test  MT8F,  Q0,  as  defined  in  MIL-STD-781C,  "Reliability 
Design  Qualification  and  Production  Acceptance  Tests:  Exponential  Distribu¬ 
tion")  and  for  an  undesirable  MTBF  (the  lower  test  MTBF,  9-j,  as  defined  in  MIL- 
STD-781C).  He  then  selects  a  test  plan  from  MIL-STD-781  which  satisfies  these 
risks,  or,  if  he  wants  to  bother,  creates  one  with  the  aid  of  various  statisti¬ 
cal  tools  such  as  the  Wald  sequential  test  formulas,  or  the  Poisson  and  chi- 
square  tables.  He  must  also  determine  the  test  environment  and  monitoring 
procedures,  a  non-trivlal  task  fortunately  outside  the  scope  of  the  report. 
Completing  these  tasks,  he  has  the  required  test  plan.  As  Figure  1  shows,  any 
prior  data  which  may  exist  is  completely  ignored  by  the  process. 

Figure  2  shows  the  process  of  creating  a  Bayesian  test.  The  responsible 
engineer  must  now  select  the  type  of  risks  he  desires  as  well  as  their  values. 
To  use  the  prior  data  he  must  analyze  it  and  extract  the  statistical  parameters 
of  interest.  These  parameters  with  his  selection  of  risk  types  and  values  are 
used  to  structure  the  test  plan.  Adding  the  test  environment  and  monitoring 
procedures,  as  before,  completes  the  job. 


6 


8 


At  this  point  the  reader  may  wonder  If  it's  worth  the  effort.  Our  intent 
is  of  course  to  reduce  the  effort,  but  even  a  difficult  effort  might  be  worth¬ 
while.  Advantages  of  the  Bayesian  approach  are: 

(a)  A  favorable  prior  will  permit  a  shorter  test  with  attendant  savings 
in  dollars  and  time.  For  example,  a  commercial  computer  may  be  purchased  for  a 
military  conmand,  control  and  comnunications  system.  If  it  has  a  good  history 
based  on  solid  and  pertinent  data,  is  it  worth  the  time  and  cost  of  a  classical 
test  when  a  Bayesian  test  can  provide  the  results  quicker?  The  only  other 
option  is  a  complete  waiver  of  testing,  which  is  not  recommended,  since  a 
particular  unit  may  be  a  lemon  even  if  the  average  unit  is  great. 

(b)  Used  for  production  reliability  control,  each  test  performed  can  use 
all  the  previous  test  data  as  prior  information.  If  a  downward  trend  is 
indicated,  the  test  plans  will  become  more  stringent.  If  the  trend  is  towards 
improved  reliability,  the  tests  become  easier.  Hence,  the  test  severity  is 
self-regulating,  providing  the  proper  test  length  and  accept  criteria  for  the 
data  available  at  all  times.  The  alternatives  are  low  risk  classical  tests 
with  greater  cost  penalties  or  high  risk  classical  tests,  which  cost  less,  but 
could  be  too  liberal  in  a  declining  reliability  trend. 

(c)  Using  classical  tests  for  production  control  compounds  the  producers 
risks, f  Assuming  his  equipment  were  of  a  constant  high  quality  such  that  it 
would  pass  a  classical  test  with  10%  risk  (90%  probability  of  acceptance),  and 
four  tests  were  scheduled  during  production,  his  probability  of  passing  all 
four  tests  would  be  (.90)^  -  .6481,  which  is  getting  uncomfortable.  With  a 
Bayesian  approach,  continued  high  quality  would  reduce  the  risk  of  rejecting 
good  equipment  as  it  would  successively  reduce  the  stringency  of  the  test. 

(d)  The  Bayesian  approach  allows  the  use  of  better  risk  definitions.  For 
example,  suppose  a  classical  test  provides  a  5%  probability  that  a  poor  MTBF 
would  be  accepted.  Suppose  all  equipment  produced  had  a  poor  MTBF.  Only  5%  of 
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the  units  tested  would  be  accepted,  but  TOG*  of  the  accepted  units  would  be 
poor.  With  Bayesian  test  plans,  instead  of  the  classical  consumers  risk,  a 
posterior  risk  can  be  used.  This  is  defined  as  the  percent  of  accepted 
equipment  which  is  poor,  which  is  obviously  a  better  choice  in  an  environment 
of  generally  poor  quality. 

The  next  sections  will  look  at  the  various  steps  in  the  Bayesian. test 
generation  process,  and  will  try  to  provide  a  simple  approach  to  each. 

3.  SELECTING  TEST  RISKS 

Classical  reliability  test  plans  are  based  or,  two  statistical  risks. 

The  producers  risk  (a)  defined  as  the  probability  of  rejecting  a  system 
whose  MTBF  is  equal  to  a  "desired"  value  eQ.  The  "desired"  value  has  been 
called  the  “specified"  value  and  is  referred  to  in  MIL-STD-781C  as  the  upper 
test  limit. 

Mathematically:  a  =  P(R|©0)  =  1  -  P(A|©0)  (3.1) 

where  P ( R | ©Q )  is  the  probability  of  rejection  given  that  0Q  is  the  true 
MTBF  of  the  test  sample,  and 

P f A | ©Q )  is  the  probability  of  acceptance. 

The  consumers  risk  (0)  is  the  probability  of  accepting  a  system  whose 
MTBF  is  equal  to  an  "undesired"  value  9^.  The  "undesired"  value  has  been 
called  the  "minimum  acceptable"  value  and  is  currently  labeled  in  MIL-STD'781C 
as  the  lower  test  limit. 

or:  e  =  P(A(01)  (3.2) 

where  P(A|0^)  is  the  probability  of  accepting  a  sample  with  a  true  MTBF 

of  0, . 
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The  need  for  0Q  and  0^  stems  from  the  fact  that  reliability  tests  are 
basically  designed  to  discriminate  between  two  alternative  hypotheses  (equip¬ 
ment  is  good  vs  equipment  is  bad),  which  must  be  numerically  defined.  Figure  3 
is  an  operating  characteristic  curve  which  plots  the  probability  of  acceptance 
against  the  true  MTBF  of  the  test  sample  for  a  typical  reliability  test.  The 
producer's  risk  (the  probability  of  rejecting  an  equipment  with  true  MTBF  of  0Q) 
is  one  minus  the  distance  from  the  horizontal  axis  is  to  the  point  on  the  curve 
directly  above  0Q.  The  consumer  risk  is  simply  the  distance  from  the 
horizontal  axis  to  the  curve  above  0-j.  Note  the  shaded  areas  labelled  a  and  B. 
These  represent  respectively  the  probability  of  rejecting  an  equipment  with  0Q 
or  higher  MTBF  and  the  probability  of  accepting  an  equipment  with  0^  or  lower 
MTBF.  These  risks  are  called  the  average  risks  and  designated  by  o  and  B.  They 
probably  more  accurately  reflect  the  desires  of  the  test  engineer,  but  cannot 
be  computed  without  the  analysis  of  prior  data  (or  an  assumption  of  the  nature 
of  the  population  of  the  equipment  to  be  tested). 

The  average  producers  risk  (5)  can  be  defined  as: 

a  =  number  of  goo J  systems  tested  that  are  rejected 
to ta’l  number  of  good  systems  tested 


Similarly: 


R  =  number  of  bad  systems  that  are  accepted 
total  number  of  bad  systems  tested 


1 


TRUE  MTBF 

OPERATING  CHARACTERISTIC  CURVE 


Obviously,  to  compute  a  and  5  requires  a  knowledge  of  the  total  number  of  good 
and  bad  systems,  and  hence  cannot  be  done  with  classical  procedures.  Using 
Bayesian  procedures,  if  we  can  define  a  probability  density  function  g(6)  for 
the  equipment  MTBF  we  can  define  the  risks  mathematically  as  follows: 


Where  P(R | 0)  =  probability  of  rejecting  an  equipment  with  MTBF  of  0 

(JO 

-  X* p  ‘R^g9  de  *3,3) 


X 


o_ 


g(0)  d  0 


06 


or 


00 

a  =  /g  P(R|0)  g(9|9  >.e0)  d0 


similarly: 

B-X 


P ( A 1 0 )  g  (0|e  <9^(1  0 


(3.4) 


(3.5) 


where  P( A ( 0 )  is  the  probabil Ity  of  accepting  a  unit  with  MTBF  of  9. 


Another  set  of  risks  possible  with  Bayesian  approaches  is  the  posterior 
risks  (a*,e*).  In  the  frequency  sense: 


a*  =  number  of  good  systems  among  the  rejected  system 
total  number  of  rejected  systems 


6*  =  number  of  bad  systems  among  the  accepted  systems 
total  number  of  accepted  systems 


mathematically: 


oo 

S  P  (R|0)  g  (0)  d  9 


7  p  (R|9)  9  (8)  d  0 

— 'o 

0, 

y  P(A|8)  g  (6)  d  6 


6*  ■  -°- 


f  P(A|e>  g  (9)  d  e 
■'o 

Another  risk  of  possible  interest  is  the  probability  of 
P(R).  This  is  a  single  number  defined  as: 

pfRv  _  Total  number  of  systems  rejected 
1  '  Total  number  of  systems  tested 


or 


oo 

P(R)  *  J  P(R|9)  g  (9)  d  0 


Note  this  risk  is  independent  of  6 
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(3.6) 


(3.7) 


rejection. 


(3.8) 


(8**) 


Other  risks  are  possible  such  as  the  alternate  posterior  consumers  risk 
defined  as  the  probability  of  an  accepted  product  being  below 


w 

6**  =  p(e  <  0oJA)  »  f  P  (0|A)  d  6 


e** 


f°  P  (A 1 0)  g  (0)  d  0 

-Q _ 

/“  P  (A 1 9)  g  (0)  d  0 


(3.9) 


(3.10) 


With  this  wealth  of  risk  options,  the  analyst  may  well  wonder  which  he  should 

use.  The  choice.  Ideally,  is  two  risk  types  which  best  fit  the  real  Interests 

of  the  producer  and  consumer.  These  interests  will  vary  according  to  the  way 
requirements  are  defined,  the  relation  between  the  number  of  units  produced  and 
the  number  procured  by  the  consumer,  and  the  wil’lngness  of  the  producer  and 
consumer  to  accept  a  prior  distribution. 

For  this  discussion  we  will  assume  that  requirements  are  defined  In  terms 
of  9Q  and  0j.  Philosophically,  0Q  should  represent  the  highest  value  of  MTBF 
believed  achievable  under  the  state-of-the-art  and  the  required  reliability 
effort  and  0^  should  be  the  lowest  value  the  consumer  would  be  willing  to 
accept  from  both  mission  success  and  supportabllity  viewpoints.  Hence,  equip¬ 
ment  with  an  MTBF  of  0Q  (or  better)  should  have  a  high  probability  of  accep¬ 
tance  and  equipment  with  0^  (or  less)  should  have  a  high  probability  of  rejec¬ 

tion. 


These  requirements  suggest  the  use  of  average  risks.  However,  the  quan¬ 
tities  produced  and  bought  can  make  other  risks  more  desirable. 

Consider  first  the  case  where  the  producer  produces  a  large  number  of 
systems  and  the  consumer  buys  a  large  number  of  systems.  A  recent  practical 
example  Is  the  AN/ARC-164  program,  where  over  10,000  airborne  UHF  radios  were 
produced  for  the  Air  Force.  Tests  based  on  prior  distribution  could  be  used  In 
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such  programs  for  production  reliability  control,  with  the  original  prior 
provided  by  data  from  the  reliability  qualification  tests,  and  the  subsequent 
priors  derived  by  combining  the  original  prior  with  the  results  of  each  produc¬ 
tion  test  performed. 

In  this  case,  the  desirable  risks  are  0*,  P(R) .  The  consumer  does  not 

have  to  limit  himself  to  0  which  tells  him  only  the  probability  that  a  unit  of 

actual  MTBF  9^  will  fail  the  test.  S  tells  him  the  probability  of  units  with  Qj 

or  less  being  accepted  which  is  more  meaningful.  However,  taking  an  extreme 

case,  suppose  all  units  produced  were  below  9,.  Then  no  matter  how  low  0  or  B 

1  * 

were,  all  accepted  items  would  be  bad.  Hence,  6  which  tells  him  the  percent 
of  accepted  units  which  will  be  bad,  is  the  most  meaningful  figure.  For 
similar  reasons,  the  consumer,  who  does  not  want  to  reject  good  units,  would 
prefer  a*  to  a  or  5  as  the  producer's  risk.  However,  the  producer  desires  to 
have  all  the  units  produced  be  acceptable  and  hence,  P(R),  representing  the 
percent  of  units  rejected  Is  his  most  pertinent  risk  criterion. 

Now,  suppose  the  producer  produces  a  large  number  of  systems  and  the 
consumer  buys  a  small  number.  An  example  of  this  could  be  the  purchase  of  a 
commercial  computer  for  use  in  an  Air  Force  Command  and  Control  system.  Here  a 
test  based  on  prior  distribution  could  be  used  for  acceptance  of  the  computer 
with  the  prior  derived  from  the  history  on  the  producer’s  products  sold  to 
previous  customers.  Another  example  Is  the  purchase  of  commercial  test  equip¬ 
ment  for  Air  Force  use. 

For  this  case,  the  desirable  risks  are:  a,  B.  Since  the  consumer  is 
procuring  a  small  quantity,  perhaps  only  one,  he  Is  concerned  with  the  risk  that 
the  lot  will  be  poor  equipment.  The  conventional  risk  0  would  do,  but  B  more 
accurately  describes  his  concern.  The  producer  could  be  Interested  in  P(R) 
Interpreted  as  the  probability  of  a  random  system  being  rejected.  However, 
since  he  has  other  customers  to  whom  he  can  sell  the  rejected  equipment  and 
presumably  does  not  want  the  stigma  of  selling  an  equipment  that  did  not  meet 
the  customer's  requirement,  a  would  better  define  his  risk. 

Another  possibility  is  that  a  specific  producer  produces  a  small  number 

of  systems  and  the  consumer  buys  a  large  number  (from  different  producers).  At 
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present,  this  case  is  not  of  great  interest.  It  arises  when  there  Is  a  need  for 
vast  quantities  of  equipment  beyond  the  capabilities  of  any  one  supplier,  such 
as  seen  during  World  War  II,  when  many  suppliers  produced  the  same  equipment 
for  the  war  effort.  The  test  would  be  used  to  accept  the  products  of  the 
individual  consumers  with  the  prior  established  from  the  first  producer  and 
updated  as  information  became  available  from  other  producers.  In  this  case, 
the  consumer  is  still  acquiring  a  large  number  of  units  and  hence,  would  prefer 
B  to  describe  his  risk.  The  producer,  however,  has  a  small  number  to  sell  and 
would  find  a  his  most  meaningful  risk. 

Finally,  suppose  the  producer  produces  a  small  number  of  systems  and  the 
consumer  buys  a  small  number.  This  is  a  frequent  case  In  Air  Force  applica¬ 
tions.  Ground  systems  can  be  procured  in  quantities  as  small  as  one  (e.g.,  the 
AN/FPS-85  radar),  and  Command  and  Control  systems  are  typically  composed  of  a 
few  (although  often  large)  components.  In  space  applications  even  an  extensive 
satellite  system  is  composed  of  a  statistically  small  population,  and  even 
airplane  systems  can  be  provided  in  small  quantities  (e.g.,  the  Advanced  Air¬ 
borne  C .  mand  Post).  In  these  cases,  tests  based  on  prior  distribution  could 
be  used  for  acceptance  of  the  equipment,  if  a  srltable  prior  exists.  However, 
a  small  production,  especially  of  large  components,  implies  a  small  pre-pro¬ 
duction  test  sample  and  the  prior  may  be  hard  to  come  by.  An  exception  would  be 
the  purchase  of  a  small  lot  of  equipment  to  add  to  an  existing  Inventory  (e.g., 
the  addition  of  a  new  site  to  an  existing  radar  system)  where  sufficient 
previous  test  data  should  be  available.  If  a  prior  exists  and  Is  acceptable  to 
both,  a  5  would  apply  since  both  producer  and  consumer  are  concerned  with  a 
small  lot. 

The  preceding  discussion  assumes  a  prior  exists  and  is  acceptable  to 
both  producer  and  consumer.  If  no  prior  exists,  there  Is  no  choice  but  to 
revert  back  to  the  classical  risks  (a  B)- 

Other  possibilities  include  the  acceptance  of  different  priors  by  pro¬ 
ducer's  S  consumers  and  the  acceptance  of  a  prior  by  one  but  not  the  other.  It 
is  possible  to  structure  test  plans  where  producer's  and  consumer's  risks  are 
based  on  different  priors  or  to  create  plans  where  one  risk  is  based  on  a  prior 

and  the  other  is  a  classical  risk.  These  hybrid  tests  should  be  acceptable  to 

17 


'  ■  ■■  *  -V 


both  producer  and  consumer,  since  each  one  can  have  the  risks  he  wants.  For 
example,  when  the  consumer  has  a  orior  and  the  producer  does  not,  the  producer 
must  fall  back  on  the  conventional  risk  ct.  The  consumer  can  specify  a  test 
using,  say,  $  as  his  risk.  Since  the  contractor  has  the  same  risk  as  the 
conventional  test  would  provide,  his  lack  of  agreement  on  the  prior  should  not 
affect  his  willingness  to  accept  the  test.  Where  tiiere  is  no  agreement  on  the 
prior,  a  test  may  be  specified  using  risks  based  on  the  different  priors.  This 
should  be  a  rare  situation  as  test  plans  are  specified  by  the  consumer  before 
contract  award,  but  it  could  conceivably  arise  in  negotiations  for  a  contract. 
Whether  or  not  the  test  provides  any  benefits  in  reduced  test  time  depends  on 
the  prior. 

While  such  hybrid  approaches  are  possible,  they  are  by  no  means  simple. 
Since  the  object  of  this  report  Is  to  simplify  the  use  of  Bayesian  test 
techniques,  our  final  recommendations  will  not  include  hybrid  test  approaches. 

The  following  matrix  may  be  used  to  select  the  most  desirable  test  risks 
as  a  function  of  equipment  quantities. 

Large  Small 
Production  Production 


Large  Small  Large  Small 

Buy  Buy  Buy  Buy 


Producers  P(R)  a  a 

Risks 


Consumers 

Risks 


B  B  B  6 


Finally,  note  that  while  the  risks  shown  In  the  matrix  are  the  most 
desirable.  In  a  practical  situation  almost  any  combination  of  risks  would  still 
be  meaningful.  The  simplest  approach  would  be  to  use  the  average  risks  aj3 
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which  are  the  closest  to  the  classical  risks.  In  fact,  the  classical  risks  are 
really  a  surrogate  for  the  average  risks,  used  because  the  classical  approach 
permits  no  other. 

4.  ESTIMATING  THE  PRIOR 

At  this  time,  two  points  must  be  established.  First,  the  statistical 
assumptions  on  which  this  procedure  is  based  and,  secondly,  the  philosophy  of 
using  priors. 

There  are  two  statistical  assianptions  underlying  the  development  in  this 
report.  These  are: 

(a)  The  prior  distribution  is  an  inverted  gamma. 

(b)  The  times  between  failures  .follow  an  exponential  distribution 
(i.e.,  a  constant  failure  rate  exists). 

The  inverted  ganrna  prior  density  is  assumed  because  It  is  a  flexible 
distribution  capable  of  representing  many  practical  situations.  It  has  proper¬ 
ties  facilitating  mathematical  manipulation,  and,  most  Importantly,  it  seems 
to  fit  the  real  world.  (Reference  RADC-TR-76-294,  Vol  III  "Reliability  Accept¬ 
ance  Sampling  Plans  Based  Upon  Prior  Distribution,  Implications  and  Determin¬ 
ation  of  the  Prior  Distribution"), 

An  inverted  gamma  distribution  is  illustrated  in  Figure  4.  Its  formula 
is: 

/  Q  -(X  +  l)e'Y/9 

g(0)  = -  (4.1) 

t  r  (X) 
where  6,  y  ,X  >o 


properties  are: 


mean  = 


variance  = 


A  -  1 

2 

y 


mode  = 


A  +  1 
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The  constant  failure  rate  is  the  standard  assumption  used  in  reliability 
demonstrations.  It  also  has  properties  facilitating  its  mathematical  hand¬ 
ling.  Of  late,  there  have  been  some  questions  raised  as  to  its  portrayal  of 
the  real  world.  The  alternate  proposal  is  most  often  a  decreasing  failure  rate 
with  time,  with  the  Ueibull  distribution  replacing  the  exponential  in  describ¬ 
ing  time  to  failure.  While  these  questions  remain  unresolved  the  exponential 
is  still  the  standard  for  reliability  demonstration  and  its  use  does  represent 
a  conservative  approach  (i.e.,  assuming  a  constant  failure  rate  when  a  decreas¬ 
ing  failure  rate  actually  exists,  would  lead  to  somewhat  pessimistic  conclus¬ 
ions).  Since  equipment  in  actual  usage  in  general  appears  to  have  a  lower 
reliability  than  indicated  by  reliability  tests,  a  conservative  approach  to 
the  test  has  merit. 

On  the  philosophy  of  using  priors,  let  it  be  clearly  understood  that  in 
this  procedure  only  actual  test  data  is  considered  a  usable  prior,  and  only 
then  if  it  exists  in  sufficient  quantity.  We  do  not  subscribe  to  "subjective" 
priors  (I.e.,  what  one  believes  the  prior  is,  without  data)  or  the  use  of 
reliability  predictions  as  priors.  (The  prediction  establishes  a  baseline 
which  the  equipment  may  or  may  not  have  achieved).  Hence,  only  hard  data  may 
be  used  for  the  prior  and  a  reversion  to  classical  testing  made  when  sufficient 
hard  data  is  not  available. 

Considering  the  above,  the  task  of  estimating  the  prior  consists  of 
fitting  available  data  to  an  inverted  ganna  distribution  and  solving  for  the 
parameters  of  the  distribution. 

The  analyst's  job  is  to  determine  the  X  and  y  of  the  formula  that  fits  his 
data.  This  Is  simple  to  say,  but  much  harder  to  realize. 

RADC-TR-76-294,  Vol  III,  referenced  earlier  examines  this  problem  and  var¬ 
ious  solutions.  The  report  recommends  the  use  of  a  maximum  likelihood  estima¬ 
tion  procedure  for  determing  X  and  y  from  the  prior.  This  procedure  requires 
the  use  of  a  computer  program  which  is  provided.  A  simpler  technique,  called 


the  Generalized  Maximum  likelihood  Method  was  developed  later  and  published  in 
RADC-TR-79-1 21 ."Application  of  Bayesian  Techniques  to  Reliability  Demonstra¬ 
tion,  Estimation  and  Updating  of  the  Prior  Distribution." 


The  procedure  is  based  on  the  function 

n 

W  (y)  s  y  ctj 


_ _  £  in  o  +  Ld 

j>l  My)  +  j  -  l  i  =  l  y 


(4.2) 


whefl 


X  (Y) 


rt 

i  =  1 


ri 

TT^y 


(4.3) 


z 

i  =  1 


T. 

l 


Ti 


and 


oi  =  Z-  ^  (i.e.  add  one  for  every  R^>j) 


ri 


(4.4) 


r.  =  number  of  failures  in  the  ith  test  sample 
T,.  =  total  test  time  of  the  i**1  sample 

This  is  illustrated  in  Figure  5. 

The  analyst’s  job  is  to  find  the  value  of  y  where  W  crosses  the  axis 
(W  =  0).  This  is  done  by  an  iterative  method,  as  follows: 

Step  1  -  a  determination  is  made  on  whether  or  not  the  data  is  sufficient 
to  establish  a  prior.  The  conditions  for  a  solution  are: 

r<—  Z  (ri  "  ~  Ti)Z  (4.5) 

n  i  =  1  T 

where  n  is  the  number  of  test  samples,  r  and  T  are  the  mean  number  of  failures 
and  mean  test  time  for  the  sample,  r^  and  T^  are  the  number  of  failures  and 
test  time  of  the  i*^  unit. 
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If  this  inequality  does  not  hold,  data  Is  Insufficient  to  establish  a  prior. 


Step  2  -  An  arbitrary  value  for  y  is  selected  and  the  equation  for  W 
solved.  If  W  is  a  negative  number,  a  smaller  value  of  y  must  be  selected  such 
as  trying  half  the  first  value  until  a  positive  W  is  obtained.  This  is  because 
the  third  and  successive  values  of  y  are  extrapolated  and  if  the  first  two  are 
obtained  where  W  is  negative  a  final  gamma  of  negative  or  Infinite  value  can 
result  (see  tangent  lines  for  yQ  &  y ^  on  Figure  5).  If  the  value  of  W  is 
positive  with  the  first  selection  of  y  then  a  higher  value  of  y  is  selected 
(try  twice  the  first  value)  and  W  computed  again. 

Step  3  -  From  the  first  two  values  of  y  a  third  value  is  extrapolated  by 

Y3  =  Yj  -  (y 2  -  Yi)  W  (y2)  (^-6) 

w  (y  2)-  w  (Yl) 

W  is  computed  for  y3.  Y3  and  Y2  v>ien  become  Y2  and  Yj  respectively,  for  the 
next  iteration. 

Step  4.  The  process  continues  until  the  change  In  y  Is  negligible  or  W 
is  satisfactorily  close  to  zero. 

Step  5  -  X  is  computed  from  equation  4.3. 

This  procedure  can  be  done  on  a  pocket  computer  but  can  be  tedious.  For 
those  with  access  to  a  computer  terminal  the  following  FORTRAN  program  may  be 
helpful . 

With  the  parameters  of  the  prior  on  hand,  the  analyst  is  ready  for  the  next 
step,  structuring  the  test  plan. 
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FORTRAN  PROGRAM  FOR  COMPUTING  A  PRIOR 


dimension  r (1000) ,t(1000) ,a(ie00) 

******  input  data  in  interactive  mode  ****** 
print,  "enter  number  of  units  tested" 
read,  n 
xn«n 

print,  "enter  number  of  failures,  test  time  for  each  unit;  one  unit  at  a  time’’ 
do  10  nl-l,n 
read,  r(nl),t(nl) 

10  continue 

******  test  for  suffici-  icy  of  data  ****** 

******RMXVTR-79-121, formula  3.3-1  ****** 
suni»0;sumt«0 
do  20  n2-l,n 
suror»sunr+r  (n2) 

20  continue 
do  30  n3»l,n 
aumt-sunt+t (n3) 

30  continue 

rbar-sumr/n  jtbar-sumt/n 

sc*0 

do  40  n4«l,n 

scl« (r (n4) - (rbar/tbar ) *t (n4) ) **2 

40  SC-SC+8C1 

sc-sc/n 

print,  "rbar  »",rbar 
print,  "check  «",sc 
if  (rbar.lt.se)  go  to  50 

print,  "insufficient  data  -  use  standard  test" 
stop 

50  continue 

print,  "prior  acceptable" 

******  solve  for  lambda  &  garma  by  eq.  3.2.1  of  Rt*DC-ra-79-121 ****** 

******  step  1.  determine  values  of  alpha  ****** 
do  60  j-1,1000 
a(j)-0. 
do  70  jl*l,n 

If  (r(jl).ge.j)  a(j)«a(j)+l 
70  continue 

if  (a(j) .eq.0.0)  go  to  90 
60  continue 
80  limj-j-1 

******  step  2.  determine  lambda  &  gamma  by  iteration  ****** 
gam-1000 

call  solve(n,w,gam,amgm,tok,limj,r,t,a} 
if  (tok.gt,0.0)  go  to  90 
gaml-gam 
wl-w 

if  (w.lt.0.0)gam-gan/2.0 
if  (w.gt .  0 . 0)  gam-2 . 0*gam 


-r  U  - ■ 
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100  continue 

call  solve  (n,w, gam, amgpn,tok,limj ,r,t,a) 
if  (tok.gt.0.0)  go  to  90 

******  th£S  extrapolates  a  new  value  of  gemma  ****** 

gamn-gam-w*  (gam-gaml )  /  (w-wl ) 

garni  »gain 

wl=w 

gam-gamn 

go  to  100 

90  print, "lambda  =  ",amgm 
print,  "gamma  *  ",gam 
end 

subroutine  solve{n,w,gam,anqm,tok,limj,r,t,a) 
dimension  r (1000) ,t (1000) ,a (1000) 
save 

******this  finds  the  value  of  w  for  an  assumed  gamma  ****** 
******  gtep  i,  find  lambda  as  a  function  of  gamma  ****** 
amgmd-0 . 0 ;  amgmn-0 . 0 
do  15  i«l,n 

angmd=*angmd+t(i)/(t(i)+gam) 

15  continue 
do  25  i2»l,n 

amgnuvamgmn+r  ( i2)  /  ( t  ( i2)  4gam) 

25  continue 
angre-gam*  ( amgmn/amgmd ) 

******  step  2.  compute  w  ****** 
w2«0.0;wl*0.0 
do  35  i3-l,n 
w2-w2+alog I 1+t (i3) /g am) 

35  continue 
do  45  i4*l,limj 
wl»wl+a ( i4) / { aragm+i4-l ) 

45  continue 
w**wl-w2 

******  step  3.  test  for  w  ■  0  ****** 

if  (w.gt. 0.000001. or .w. It. -0.000001}  go  to  16 

tok-1.0 

return 

16  tok=-l . 0 
return 

end 
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STRUCTURING  TEST  PLANS 


Test  plans  are  designed  to  permit  accept/reject  decisions  satisfying  de¬ 
signated  producer's  and  consumer's  risks.  For  each  curve  an  operating  charac¬ 
teristic  curve  (see  Figure  3)  can  be  drawn.  Under  the  assumed  exponential 

failure  distribution  the  curve  can  be  drawn  from  the  expression: 

« 

P(A|0)  =  l  e  ^  (5.1) 

r=o  r! 

where  P (A J G)  is  the  probability  of  acceptance  for  a  given  MTBF  (0) 

T  Is  the  total  test  time  (the  sum  of  the  test  times  on  each  sample) 
r*  is  the  number  of  failures  allowed  for  acceptance  (r  +  1  ■  reject) 

This  is  a  Poisson  distribution  and  for  classical  test  plans,  0  is  set  to 
©1  to  compute  the  consumers  risk.  The  producer's  risk  is  computed  by  setting  0 
to  0O  and  subtracting  the  results  from  1.  (Probability  of  rejection  *  1  - 
probability  of  acceptance).  Synthesis  of  the  test  involves  setting  the  prob- 
ability  to  a  desired  value,  and  for  various  values  of  r  ,  solving  for  T/0. 
Since  0  Is  fixed,  this  procedure  gives  a  series  of  tests  defined  by  pairs  of  r 
and  T  values  yielding  the  same  risk.  To  provide  a  test  satisfying  desired 
producer's  and  consumer's  risks,  two  series  of  tests  are  produced,  one  for  the 
desired  producer's  risk  and  one  for  the  desired  consumer's  risk.  A  pair  of  r 
and  T  values  in  one  series  which  matches  (or  is  satisfactorily  close  to)  a  pair 
In  the  other  series,  defines  a  fixed  test  that  satisfies  both  risks.  MIL-STD- 
781  provides  a  tabulation  of  classical  fixed  time  tests  for  various  risks  (as 
well  as  sequential  tests,  which  are  more  sophisticated  and  permit  earlier 
decisions  than  fixed  time  tests  for  the  same  risks). 

An  analogous  procedure  is  used  to  structure  Bayesian  fixed  time  tests.  Of 
necessity.  It  becomes  much  more  complicated.  For  example,  equation  3-7  gives 
the  posterior  consumer's  risk  as: 
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(5.2) 


y  P  (A|9)g(0)d9 

8*  =  _° _ 

?  P(A|0)g{0)d9 
J  o 

substituting  equation  5.1  for  P(A jO)  and  4.1  for  g'(9)  we  get: 


,  r* 

z 

[  r=o 

e‘T/9(T/9)r-j 

.  A 

Y 

e-(x+i)e-Y/ed0  (5-3) 

r!  J 

r(A) 

00 

/  / 

.  r* 

e'T/0(T/0)r* 

A 

Y 

g-(X+l )e-Y/9dg 

0  1 

l— 

r=o 

rf  J 

r(A) 

This  formidable  expression  is  converted  to  fixed  time  test  plans  (test  length 
T*,  accept  with  r*  or  less  failures)  by  the  following  process: 

Step  1  -  The  following  normalizing  functions  are  substituted  into  the  risk 
equations: 

t*  -  T/e0 
Y*  =  ve0 
k  *  e0/e1 

0  =  0/9o 

Equation  5.3  then  becomes: 


3*= 


^USSL }  .-O+iVyV*  di 

r!  J  r(A)  V 


e’TVl!>(T*/0)S  Y*X  -(X+l)e  -Y*/<U 

— FI - 7  rw 


(5.4) 
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DETERMINING  ACCEPTABLE  TEST  PLANS 


Step  2  -  since  y*,A,  K,  (I,  and  B*  are  known,  acceptable  values  of  T*  and  r* 
can  be  found,  if  a  solution  exists.  Equation  5.3  computes  a  consumer's  risk,, 
similar  solutions  can  be  found  for  the  desired  producer's  risk.  Figure  6 
shows  a  generalized  plot  relating  r *  and  T*  for  an  arbitrary  consumer's  risk 
and  a  producer's  risk.  The  point  at  which  the  two  curves  cross  is  the  test 
plan  satisfying  both  risks.  This  may  be  found  by  solving  simultaneously  the 
equation  for  producer's  and  consumer's  risk. 

An  obvious  problem  with  the  preceding  approach  is  that  the  two  curves 
will  rarely  cross  at  an  integral  number  for  r*.  (For  example,  what  does  one 
do  with  an  accept  number  of  1.76  failures?).  One  solution  is  to  select  the 
first  integer  reached  on  the  left  hand  curve  above  the  intersection.  This 
gives  a  test  plan  with  the  desired  consumer's  risk  but  a  lower  than 
established  producer's  risk.  In  compensation,  it  provides  the  shortest 
possible  test  time  not  exceeding  either  established  risk.  Plots  of  risk  can 
be  obtained  from  a  computer  program  presented  in  RADC-TR-76-294  Vol  IV. 
However,  to  simplify  applications  of  the  procedure,  RADC-TR- 78-241  "Design 
of  Reliability  Test  Plans  Based  Upon  Prior  Distribution"  provides  tabulations 
of  test  plans,  just  as  MIL-STD-781  tabulates  classical  fixed  time  tests. 

Test  plans  are  given  for  the  following  combination  of  risk  and  discrimination 
ratios  (i.e.,  Qq/Q-j). 


1 .  a 

= 

8*  =  0.10 

K  =  1.5 

2.  a 

= 

e*  =  o.io 

K  =  2.0 

3.  a 

= 

8*  =  0.10 

K  =  3.0 

4.  a 

= 

8*  =  0.15 

K  =  1.5 

5.  a 

= 

8*  =  0.15 

K  =  2.0 

6.  a 

= 

8*  =  0.15 

K  =  3.0 

7.  a 

= 

8*  =  0.20 

K  =  1.5 

8.  a 

= 

8*  =  0.20 

K  =  2.0 

9.  a 

- 

8*  =  0.20 

K  =  3.0 

TO.oi* 

- 

8*  =  0.10 

K  =  1.5 

11. a* 

= 

8*  =  0.10 

K  ■  2.0 

12. a* 

= 

8*  =  0.10 

K  =  3.0 

13.  a* 

= 

8*  =  0.15 

K  =  1.5 

14-  a* 

= 

8*  =  0.15 

K  =  2.0 

15.  a* 

= 

8*  =  0.15 

K  ■  3.0 

16.  ct* 

= 

8*  «  0.20 

K  =  1.5 

17.  a* 

- ' 

8*  =  0.20 

K  =  2.0 

18.  a* 

= 

8*  =  0.20 

K  =  3.0 

19.  a 

= 

B  =  0.10 

K  =  1.5 

20.  a 

= 

B  -  0.10 

K  =  2.0 

2  i .  a 

= 

§  =  0.15 

K  *  1.5 

22.  a 

= 

B  =  0.15 

K  *  2.0 

23.  ci 

= 

8  =  0.20 

K  *  i.S 

24.  a 

= 

8  =  0.20 

K  =  2.0 

Unlike  MIl-STO-781  each  test  plan  In  RADC-TR-78-Z41  is  not  one  fixed  set  of 
values  Hrr  and  t  .  Rather,  it  is  a  table  of  values  for  different  combina¬ 
tions  of  T  and  i  idlfch  each  add  another  dimension  to  the  problem.  Appendix  I 
provides  a  reprint  of  RADC-TR-78-Z41  table  6.20  for  5  .  6  .  0.10  k  >  2  o 
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These  tables  will  make  test  selection  easy,  but  present  some  problems. 
First,  the  case  of  Interest  may  not  be  among  those  tabulated,  (e.g.,  the  tables 
do  not  include  any  cases  where  the  producer's  risk  Is  the  overall  probability 
of  rejection,  nor  are  discrimination  ratios  greater  than  two  used  for  a  B  test 
plans).  However,  the  tables  probably  represent  a  useful  range,  since  one  can 
always  use  one  of  the  type  of  risks  presented  and  discrimination  ratios  greater 
than  two  are  philosophically  questionable.  (If  0Q  represents  the  achievable 
state  of  the  art,  why  should  01  be  set  at  less  than  half  that  value?) 

Another  problem  with  the  tables  is  the  r*.  t*  entries  of  0,0.00  repre¬ 
senting  accept  without  testing  and  ***  representing  reject  without  testing. 
Neither  is  really  desirable.  These  cen  be  overcome  by  designating  a  minimum 
test  to  be  used  when  the  prior  Is  favorable  enough  for  a  0,0.00  entry  and 
reverting  to  a  designated  classical  test  when  the  prior  is  unfavorable  enough 
for  a  ***  entry.  Reversion  to  a  classical  test  is  also  a  possible  alternative 
when  the  Bayesian  test  would  be  longer  than  an  acceptable  classical  test. 

The  final,  and  most  serious  problem,  is  the  possibility  that  the  prior 
data  will  extend  beyond  the  table  limits,  (l.e.,  X  >  4.0  or  y*>  3.45).  In  this 
situation,  one  could  revert  back  to  the  computer  programs  of  RADC-TR-76-294  or 
to  the  original  formulas  (in  Section  3).  There  Is  a  slmpller  way,  however, 
which  is  much  easier,  but  which  may  horrify  statistical  purists. 

The  method  basically  converts  a  prior  distribution  oucslde  the  table 
limits  to  one  with  the  same  mean  within  the  table  limits. 

The  mean  of  the  prior  is 

T 

u  * — xn 


(5.6) 
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6. 


PUTTIK6  IT  ALL  TOGETHER: 


Applying  the  preceding  discussion  to  a  practical  situation  starts  with  the 
determination  that  a  Bayesian  test  is  feasible  (e.g. ,  for  production  sampling 
or  for  testing  a  commercial  unit  with  available  field  data).  If  the  test  is 
feasible  the  manager  must  determine  what  type  of  risks  and  what  risk  value  he 
desires.  He  must  also  define  some  default  conditions.  These  are: 

(a)  A  classical  test  plan  to  be  used  if  the  prior  is  inadequate,  is  so 
unfavorable  as  to  indicate  reject  without  testing,  or  results  in  a  test  longer 
than  an  acceptable  classical  test. 

(b)  A  minimum  test  to  be  used  if  the  prior  is  so  favorable  as  to  indicate 
accept  without  testing. 

(c)  A  requirement  for  corrective  action  and  retest  if  the  test  is  failed. 
This  is  a  necessity  for  all  test  plans.  A  unique  twist  in  Bayes  testing  would 
be  the  requirement  for  the  first  retest  after  corrective  action  to  be  a  classi¬ 
cal  test  which  would  then  become  the  prior  for  any  further  tests.  The  previous 
prior  would  be  discarded  since  the  corrective  action  should  change  the  equip¬ 
ment. 


These  determinations  are  then  put  into  the  contract  provisions,  which 
should  also  require  the  customer's  approval  of  all  data  used  to  structure  the 
prior. 


In  implementation,  the  data  is  first  analyzed  for  sufficiency.  If  not 
sufficient,  the  classical  test  is  used.  If  sufficient,  the  X  and  y  are 
obtained  and  the  test  plan  generated  or  extracted  from  the  appropriate  tables. 
If  the  tables  are  used  and  X,  y  or  both  are  outside  the  table  limits,  the  prior 
is  adjusted  in  accordance  with  equations  5.7  and  5.8.  If  the  prior  1$  extreme¬ 
ly  favorable,  the  minimum  test  is  required.  If  extremely  unfavorable,  the 
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classical  test  is  mandated.  The  test  Is  then  performed.  If  successful  the 
test  data  Is  added  to  the  previous  data  and  a  new  prior  created  for  the  next 
test,  if  any.  If  the  test  fails,  corrective  action  is  performed  and  a  retest, 
using  the  classical  test,  performed.  The  results  of  the  classical  test  becomes 
the  prior  for  the  next  test,  if  any. 

Appendix  II  provides  sample  requirements  for  applying  Bayesian  reliabil¬ 
ity  testing  to  procurements  of  commercial  equipment.  Appendix  III  provides 
sample  requirements  for  applying  Bayesian  techniques  to  production  reliability 
acceptance  testing. 

7.  EPILOGUE 


Bayesian  testing  has  much  to  recommend  it.  It  is  hoped  that  this  report 
will  help  to  spur  its  practical  application.  The  author  would  appreciate  any 
feedback  from  the  readers.  Reports  of  actual  implementation  would  be  especial¬ 
ly  welcome.  Criticisms  and  problems  will  also  be  appreciated  though  probably 
not  en/joyed.  Address  all  comments  to 

ANTHONY  COPPOLA 

RADC/RBET 

Griffiss  AFB  NY  13441 

Call:  315-330-4726  or  Autovon:  587-4726 

For  those  so  inclined,  Appendix  IY  provides  a  l’sting  of  RADC  studies  in 
Bayesian  Reliability  Demonstration  for  more  detailed  study. 


Happy  Testing! 
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APPENDIX  I 


The  following  pages  are  a  reprint  of  Table  6.20  from  RADC-TR-78-241 
providing  test  plans  for  average  risks  of  10%  and  a  discrimination  ratio  of 
2.0,  over  a  range  of  values  for  the  parameters  of  the  prior  distribution. 
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APPENDIX  II 


The  following  are  sample  contractual  requirements  for  the  application  of 
Bayesian  Reliability  Tests  to  the  purchase  of  commercial  equipment.  Blanks 
represent  information  to  be  supplied  by  the  user.  Bold  face  entries  are  not 
part  of  the  sample  but  refer  to  notes  to  the  user  listed  after  the  sample. 
These  must  be  considered  by  the  user  In  Implementing  the  sample  paragraphs. 
The  sample  paragraphs  must  be  placed  in  appropriate  sections  of  the  statement 
of  work  and/or  specification  and  backed  up  by  appropriate  data  Items  In  the 
contract  data  requirements  list.  On  the  latter,  it  is  believed  that  only 
standard  data  items  (the  same  ones  used  for  classical  reliability  demonstra¬ 
tion  requirements}  are  needed  for  the  Bayesian  approach. 

x.  Reliability.  The  _  shall  be  designed  to  achieve  a 

mean  time  between  failure  (MTBF)  of  no  less  than _ .  hours.  This  figure 

is  designated  the  upper  test  MTBF  (80)  for  demonstration  purposes.  The  lower 
test  MTBF  (Oj)  is  defined  as _ hours.  (SEE  MOTE  1) 

y.  Reliability  Demonstration.  The  Reliability  of  the _ shall 

be  demonstrated  In  accordance  with  MIL-STD-781,  except  that  a  Bayesian  Reli¬ 
ability  Test  may  be  used  in  lieu  of  the  test  plans  listed  In  MIL-STD-781.  The 
test  to  be  used  shall  be  determined  as  follows: 

y.l.  The  contractor  shall  provide  all  available  data  showing  operating 
time,  number  of  failures  and  use  environment  by  Individual  Item  of  equipment 
Identical  to  the  unit  purchased.  The  procuring  activity  will  determine  what 
portion,  if  any,  of  the  data  may  be  used  to  establish  a  prior  for  the 
reliability  test  and  may  supply  additional  data,  If  available.  (See  Note  2). 

y.2.  Data  approved  by  the  procuring  activity  will  be  tested  for  adequacy 
by  the  criteria  of  formula  4.5  In  RADC-TR  - _ .  (See 
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Note  3).  If  no  approved  data  is  available,  or  If  the  data  falls  the  adequacy 

criteria,  a  Bayesian  test  will  not  be  used  and  test  plan  _  of  MIL-STD-781 

will  be  used  instead. 

y.3.  If  adequate  approved  data  is  available,  formulas  4.2  -  4.4  of  RADC- 

TR- _  (See  Note  3)  will  be  used  to  determine  the  prior  parameters  (X)  and 

(y).  The  test  parameter  (y*)  will  be  computed  by  the  formula:  y*  ■  y/0Q. 

y.4.  The  values  of  (y*)  and  (X)  will  be  used  with  table _ of  RADC-TR- 

78-241  (See  Note  4)  to  determine  the  total  test  time  (T*)  in  multiples  of  0O  and 
the  allowable  value  of  failure  (r*).  In  using  the  table,  the  following 
procedures  will  be  followed: 

y.4.1.  If  the  value  of  (y*),  (X)  or  both  fall  outside  the  limits  of  the 

table,  formulas  5.7  and  5.8  of  RADC-TR- _  (SEE  NOTE  3)  will  be  used  to 

compute  values  within  the  table  limits,  which  shall  be  used  in  lieu  of  the 
original  values. 

y.4. 2.  A  table  entry  of  (**♦**)  shall  negate  the  use  of  a  Bayesian  test, 
and  test  plan  _ of  MIL-STD-781  shall  be  used  instead  (See  Note  5). 

y.4. 3.  A  table  entry  of  (0,0.0)  for  (r*,T*)  shall  not  result  In  acceptance 

without  testing.  Instead  the  unit  will  be  tested  for  _  hours  with 

_  failures  allowed  (See  Note  6). 

y.5.  In  the  event  the  test  Is  not  successful,  the  contractor  shall 
propose  appropriate  corrective  actior  in  accordance  with  para  5.9  of  MIL-STD- 

781C.  A  retest  will  be  performed  using  test  plan  _  of  MIL-STD-781  (See 

Note  7). 


y.6.  These  test  procedures,  the  test  environment  and  monitoring 
provisions  will  be  submitted  for  approval  by  the  procuring  activity  and 
modified  as  necessary  before  the  test  begins  (See  Note  8). 
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NOTES  TO  USERS 


■A 
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1.  The  ratio  of  0o/9^  specified  In  para  x,  must  match  the  tests  specified  in 
para  y.3  and  y.4.  It  must  also  match  the  default  test  in  y.2. 

2.  ;  t  is  essential  that  the  data  used  for  the  prior  be  realistic  (l.e.,  be  for 
the  same  equipment  under  the  desired  use  environment)  and  complete  (i.e., 
include  information  on  poor  units  as  well  as  good  units). 

3.  Insert  the  TR  number  assigned  to  this  report  (shown  on  the  front  cover). 

4.  The  appropriate  table  must  be  selected  to  match  the  desired  discrimination 
ratio  (Oq/Qj),  the  desired  types  of  consumers  and  producers  risks,  and  the 
desired  value  of  the  risks.  If  0Q/0j  ■  2,  and  average  risks  of  10%  are  desired, 
reference  may  be  made  to  Appendix  I  of  this  report  Instead  of  to  RAOC-TR-78- 
241,  thus  eliminating  one  reference  document. 

5.  The  same  MIl-STD-781  test  plan  referenced  in  para  y.2  should  be  referenced 
here. 

6.  This  is  the  minimum  test  and  It  is  your  option  to  select  one  that  will 
satisfy  you  that  you  haven’t  got  a  lemon.  Suggestions  might  be  to  test  for  0^ 
hours  and  allow  one  failure  (if  you  have  a  lot  of  confidence  in  the  unit)  or 
test  for  Qq  hours  and  allow  one  failure  (a  more  conservative  approach,  but 
still  more  generous  than  a  classical  test). 

7.  If  the  test  is  failed,  forget  the  Bayesian  approach  and  revert  to  the 
classical  test.  Use  the  same  test  referenced  in  y.2.  Note  that  data  item  01- 
R-7038  (Plan,  Corrective  Action)  must  be  listed  In  the  contract  data 
requirements  list.  You  may  want  some  others  also,  (See  para  30,  Appendix  F, 
MIL-STD-781C). 

8.  Oata  Item  DI-R-7033  (Plan,  Reliability  Test)  must  be  listed  in  the  contract 
data  requirements  list  (CORL).  Oata  Item  DI-R-7035  (Procedures,  Reliability 
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Tests)  will  also  be  necessary,  but  may  be  combined  with  DI-R-7033  (See  para 
5.1.4,  MII-STD-781C).  0I-R-7034  (Reports  Reliability  Test  and  Demonstration) 
Is  also  needed. 


APPENDIX  III 


The  following  are  sample  contractual  provisions  for  the  use  of  Bayesian  Tests 
for  production  reliability  control.  As  in  Appendix  II,  blanks  represent  Infor¬ 
mation  to  be  supplied  by  the  user  and  bold  face  entries  are  not  part  of  the 
sample,  but  refer  to  notes  to  the  users  listed  afterwards.  The  sample  para¬ 
graphs  must  be  placed  in  appropriate  contractual  documents  and  backed  up  by 
appropriate  data  items  in  the  contract  data  requirements  list.  It  Is  believed 
that  the  same  data  items  now  used  for  production  reliability  testing  will 

.  r 

suffice  for  the  Bayesian  approach.  One  final  note:  In  applying  this  sample, 
it  is  assumed  that  a  classical  reliability  demonstration  has  been  required  as  a 
reliability  qualification  test  and  that  9o  and  91  were  defined  for  that  test. 

x.  Reliability  acceptance  testing  will  be  performed  as  follows: 

x.l  Production  samples: _  (See  Hote  l). 

x.2  Test  Procedure:  A  Bayesian  Reliability  Testing  approach  will  be 
used  for  reliability  acceptance  as  follows: 

x.2.1  The  data  from  the  reliability  qualification  test  will  be  used  to 
establish  the  first  prior  for  the  reliability  acceptance  tests.  So  long  as  all 
the  reliability  acceptance  tests  are  successful,  the  test  data  for  all 
completed  tests  will  be  combined  with  the  reliability  qualification  test  data 
to  establish  the  prior  for  the  next  test.  Each  prior  will  be  tested  for 
adequacy  by  formula  4.5  of  RADC-TR- _  (SEE  MOTE  2).  Until  an  adequate 
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prior  Is  obtained,  the  reliability  acceptance  tests  shall  be  performed  using 

test  plan _ of  MIL-STD-781C.  (See  note  3).  When  the  prior  Is  adequate  the 

following  procedure  will  be  used: 

x.2.1.1.  Formulas  4.2  -  4.4  of  RADC-TR- _ (SEE  NOTE  2)  will  be  used 

to  determine  the  prior  parameters  (X)  and  (y).  The  test  parameter  (y*)  will  be 
computed  by  the  formula  y*  *  y/0o. 

x.2.1.2.  The  values  of  (y*)  and  (X)  will  be  used  with  table _ of 

RADC-78-241  (See  Note  4)  to  determine  the  test  time  (T*)  in  multiples  of  0Q  and 
the  allowable  number  of  failures  (r*).  In  using  the  table,  the  following 
procedures  will  be  followed: 

x. 2. 1.2.1.  If  the  value  of  (y*),  (X)  or  both  fall  outside  the  limits  of 

the  table,  formulas  5.7  and  5.8  of  RADC-TR- _ (SEE  NOTE  3).  will  be  used 

to  compute  values  within  the  table  limits,  which  shall  be  used  In  lieu  of  the 
original  values. 


x.2.1.2. 2.  A  table  entry  of  (00.0)  for  (r*,  t*)  shall  not  result  in 

acceptance  without  testing.  Instead  the  sample  will  be  tested  for  _  total 

hours  with _ failures  allowed.  (See  Note  5). 

x. 2. 1.2.2.  A  table  entry  of  (***♦*)  reflects  an  unfavorable  prior  and 
should  not  be  accessed  under  these  requirements.  Its  effect  would  be  the  same 
as  a  failed  test. 

x.2.2.  Reliability  acceptance  of  production  units  shall  cease 
whenever  a  reliability  acceptance  test  reaches  a  reject  decision.  In  such 
event : 


x.2.2.1.  The  contractor  shall  propose  corrective  action  for 
deficiencies  In  the  equipment  in  accordance  with  para  5.9  of  MIL-STD-781C. 
Following  approval  and  implementation  of  this  action,  the  reliability 
qualification  test  will  be  rerun.  On  its  successful  completion,  reliability 
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acceptance  tests  will  be  resumed  as  scheduled,  but  only  data  from  the  latest 
reliability  qualification  test  will  be  used  to  establish  the  first  prior,  and 
only  data  from  successful  reliability  acceptance  tests  following  the  latest 
reliability  qualification  test  will  be  used  to  update  the  prior.  (SEE  NOTE  6). 


NOTES  TO  USERS 


1.  Specify  how  many  samples  and  how  often  the  test  will  be  run,  (e.g.  -five 
samples  will  be  chosen  randomly  from  each  month’s  production  and  the  test  begun 
on  the  first  day  of  the  next  month.  The  test  will  run  continuously  on  all 
units,  except  for  necessary  repairs,  until  an  accept/reject  decision  is  reach¬ 
ed). 

2.  Insert  the  TR  number  assigned  to  this  report  (shown  on  the  front  cover). 

3.  Any  plan  with  the  sane  discrimination  ratio  (Oq/O^ )  as  the  reliability 
qualification  test  can  be  used,  including  a  repeat  of  the  RQT  itself.  Suggest 
a  plan  with  more  qenerous  risks  (i.e.,  if  the  RQT  used  10%  risks,  consider 
using  20%  risks)  to  reduce  test  time. 

4.  The  appropriate  table  must  be  selected  to  match  the  desired  discrimination 
ratio  (©o/©1 ) ,  and  the  desired  types  of  consumers  and  producers  risks,  and  the 
desired  value  of  the  risks.  If  Q0/Q^  =  2,  and  average  risks  of  10%  are  desired 
reference  may  be  made  to  Appendix  I  of  this  report  instead  of  to  RADC-TR-78- 
241,  thus  eliminating  one  reference  document. 

5.  This  is  the  minimum  test  used  where  all  previous  tests  were  highly  favor¬ 
able.  Your  choice  of  test.  A  suggestion  could  be:  test  for  with  one 
failure  allowed. 

6.  The  contract  data  requirements  list  must  include  appropriate  items  for  the 
test  plan.  Test  procedures,  test  report,  corrective  action  plan  and  any  other 
documentation  (e.g.,  failed  item  analysis)  you  desire.  See  para  30,  Appendix  F 
of  MIL-STD-781C  for  data  item  numbers. 


sum  suixsm  nms> 
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APPENDIX  IV 


The  following  reports  by  RADC  on  Bayesian  Reliability  Demonstration  are 
available  from: 

National  Technical  Information  Service 
Department  of  Commerce 
5285  Port  Royal  Road 
Springfield  VA  22151 

When  ordering  documents  from  NTIS,  their  reference  number  (AD  Numbers  shown  in 
parenthesis  following  the  RADC-TR  Number)  should  be  cited. 

1.  Hughes  Aircraft  Company,  "Bayesian  Reliability  Demonstration:  Phase  I  - 
Data  for  the  Priori  Distributions,"  RADC-TR-69-389  (AD-866166) 

2.  Feduccia,  A.  J.,  "A  Bayesian/Classical  Approach  to  Reliability  Demonstra¬ 
tion,"  RADC-TR-70-72  (AD-871969) 

3.  Hughes  Aircraft  Company,  "Bayesian  Reliability  Demonstration:  Phase  II  - 
Development  of  A  Priori  Distribution,"  RADC-TR-71-209  (AD-732283) 

4.  Hughes  Aircraft  Company,  "Bayesian  Reliability  Demonstration:  Phase  III  - 
Development  of  Test  Plans,"  RADC-TR-73-139  (AD-765172/2) 

5.  Syracuse  University,  "Reliability  Acceptance  Sampling  Plans  Based  Upon 
Prior  Distribution,"  RADC-TR-76-294  Vols  I-V  (Vol  I  -  AD-A033515,  Vol  II  - 
AD-A033516,  Vol  III  -  AD-A033517,  Vol  IV  -  AD-A033576,  Vol  V  -  AD-A033518) 

6.  Syracuse  University,  "Design  of  Reliability  Test  Plans  Based  Upon  Prior 
Distribution,"  RADC-TR-78-241  (AD-A064142) 

7.  SUNY,  "Application  of  Bayesian  Techniques  to  Reliability  Demonstration; 
Estimation  and  Updating  of  the  Prior  Distribution,"  RADC-TR-79-121  (AD- 
A070236) 
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SCEEE,  "Bayesian  Reliability  Theory  for  Repairable  Equipment,"  RADC-TR-80- 
30  (AD-A083009) 


